This Facial Expression Recognition with Keras is part of Coursera Project: Facial Expression Recognition with Keras. We will build and train a convolutional neural network (CNN) in Keras from scratch to recognize facial expressions. The data consists of 48x48 pixel grayscale images of faces. The objective is to classify each face based on the emotion shown in the facial expression into one of seven categories (0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral). You will use OpenCV to automatically detect faces in images and draw bounding boxes around them. Once you have trained, saved, and exported the CNN, you will directly serve the trained model predictions to a web interface and perform real-time facial expression recognition on video and image data.
This is my learning experience of data science through DeepLearning.AI. These repository contributions are part of my learning journey through my graduate program masters of applied data sciences (MADS) at University Of Michigan, DeepLearning.AI, Coursera & DataCamp. You can find my similar articles & more stories at my medium & LinkedIn profile. I am available at kaggle & github blogs & github repos. Thank you for your motivation, support & valuable feedback.
These include projects, coursework & notebook which I learned through my data science journey. They are created for reproducible & future reference purpose only. All source code, slides or screenshot are intellectual property of respective content authors. If you find these contents beneficial, kindly consider learning subscription from DeepLearning.AI Subscription, Coursera, DataCamp
Facial Expression Recognition with Keras
Practice Project: Facial Expression Recognition with Keras
# Specify the GPU device to usegpus = tf.config.list_physical_devices('GPU')if gpus:# Set the GPU memory growth to Truetry: tf.config.experimental.set_memory_growth(gpus[0], True)exceptRuntimeErroras e:print(e)
### Task 4: Create Convolution Neural Network (CNN) Model
Inspired by Goodfellow, I.J., et.al. (2013). Challenged in representation learning: A report of three machine learning contests. Neural Networks, 64, 59-63. doi:10.1016/j.neunet.2014.09.005
EMOTIONS_LIST = ["Angry", "Disgust",
"Fear", "Happy",
"Neutral", "Sad",
"Surprise"]
def __init__(self, model_json_file, model_weights_file):
# load model from JSON file
with open(model_json_file, "r") as json_file:
loaded_model_json = json_file.read()
self.loaded_model = model_from_json(loaded_model_json)
# load weights into the new model
self.loaded_model.load_weights(model_weights_file)
#self.loaded_model.compile()
#self.loaded_model._make_predict_function()
def predict_emotion(self, img):
global session
set_session(session)
self.preds = self.loaded_model.predict(img)
return FacialExpressionModel.EMOTIONS_LIST[np.argmax(self.preds)]
This code uses camera.py file to process camera to capture video or it can process provided video placed in videos folder
Here if we replace cv2.VideoCapture(‘videos/facial_exp.mkv’) with cv2.VideoCapture(0) then it will capture video stream from provided or connected camera.
import cv2 from model import FacialExpressionModel import numpy as np
facec = cv2.CascadeClassifier(‘haarcascade_frontalface_default.xml’) model = FacialExpressionModel(“model.json”, “model_weights.h5”) font = cv2.FONT_HERSHEY_SIMPLEX
class VideoCamera(object): def init(self): #self.video = cv2.VideoCapture(‘videos/facial_exp.mkv’) self.video = cv2.VideoCapture(0)
def __del__(self):
self.video.release()
# returns camera frames along with bounding boxes and predictions
def get_frame(self):
_, fr = self.video.read()
gray_fr = cv2.cvtColor(fr, cv2.COLOR_BGR2GRAY)
faces = facec.detectMultiScale(gray_fr, 1.3, 5)
for (x, y, w, h) in faces:
fc = gray_fr[y:y+h, x:x+w]
roi = cv2.resize(fc, (48, 48))
pred = model.predict_emotion(roi[np.newaxis, :, :, np.newaxis])
cv2.putText(fr, pred, (x, y), font, 1, (255, 255, 0), 2)
cv2.rectangle(fr,(x,y),(x+w,y+h),(255,0,0),2)
_, jpeg = cv2.imencode('.jpg', fr)
return jpeg.tobytes()
Below is basic flask application html to use to render an html version of emotion capture & its processing
Face expression recognition
Last but not least is the main.py routine which will initiate through terminal (in mac) or through anacoda shell in windows to launch facial expression routine
import socket
from flask import Flask, render_template, Response from camera import VideoCamera